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ABSTRACT 

We calculate the advances in near-infrared astronomy made possible through the use 
of fibre Bragg gratings to selectively remove hydroxyl emission lines from the night 
sky spectrum. Fibre Bragg gratings should remove OH lines at high resolution (R = 
10,000), with high suppression (30dB) whilst maintaining high throughput (ss 90 per 
cent) between the lines. Devices currently under construction should remove 150 lines 
in each of the J and H bands, effectively making the night sky surface brightness 
w 4 magnitudes fainter. This background reduction is greater than the improvement 
adapative optics makes over natural seeing; photonic OH suppression is at least as 
important as adaptive optics for the future of cosmology. 

We present a model of the NIR sky spectrum, and show that the interline con- 
tinuum is very faint (« 80 photons s _1 m -2 arcsec -2 /im _1 on the ecliptic plane). 
We show that OH suppression by high dispersion, i.e. 'resolving out' the skylines, 
cannot obtain the required level of sensitivity to reach the interline continuum due to 
scattering of light. The OH lines must be suppressed prior to dispersion. 

We have simulated observations employing fibre Bragg gratings of first light ob- 
jects, high redshift galaxies and cool, low-mass stars. The simulations are of complete 
end-to-end systems from object to detector. The results demonstrate that fibre Bragg 
grating OH suppression will significantly advance our knowledge in many areas of 
astrophysics, and in particular will enable rest-frame ultra-violet observations of the 
Universe at the time of first light and reionisation. 

Key words: infrared:general-instrumentation:miscellaneous-atmosphcric effects- 
cosmology: early Universe 



1 INTRODUCTION 

The future of cosmology is dependent in part on the ability 
to achieve deep observations at near-infrared (NIR) wave- 
lengths. As we strive to observe the early Universe, our ob- 
servations must be performed at increasingly longer wave- 
lengths, as the diagnostic optical spectroscopic features be- 
come more redshifted. At very high redshifts, observations in 
the optical become futile due to the redshifted Lyman limit. 
Figure 1 shows the observed wavelengths of several useful 
spectroscopic lines as a function of redshift, illustrating the 
requirement for NIR observations. 

The need for NIR observations is well documented. Over 
the last few years there have been several major reviews of 
the current status of our knowledge of the Universe, and in 
particular of gaps in our understanding, serving as a start- 
ing point for strategic plans for future observations (e.g. the 
Australian Decadal Plan for Astronomy 2006-2015 1 , the UK 

* sce@aao.gov.au 

1 http:/ /www. atnf.csiro.au/nca/DecadalPlan_web.pdf 
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Figure 1. The observed wavelength of several useful spectral lines 
as a function of redshift. The shaded areas show the wavelength 
coverage of standard zJH filters. 
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Table 1. Typical sky-brightncsscs at a good observing site. 

Surface brightness/ mag arcsec - 2 
Band Natural No OH emission 

U 21.5 

B 22.4 

V 21.7 

R 20.8 21.7 

I 19.9 21.7 

J 16.0 22.1 

H 14.0 22.2 

PPARC roadmap , the US National Research Council's 'As- 
tronomy and Astrophysics in the New Millennium' NRC 
2001 and 'Connecting Quarks to the Cosmos' Committee 
On The Physics Of The Universe et al. 2003). 

These reviews provide similar lists of the 'big questions' 
for the future of astronomy, showing a consensus of opinion 
at an international level. Some of the key issues they iden- 
tify are: What is the nature of dark energy and dark matter? 
How and when do the first stars form? How and when did 
the Universe reionise? How and when do galaxies assemble, 
and how do they evolve? What is the relationship between 
super-massive black-holes and galaxies? How do stars and 
planetary systems form? How common are planetary sys- 
tems capable of supporting life? 

The success of answering the questions above is reliant 
on NIR observations. The first half of the list depends upon 
the ability to observe the high redshift Universe, whilst the 
problems concerning planetary systems will also benefit from 
dust-penetrating NIR observations, especially since plane- 
tary sized bodies emit most of their radiation at NIR wave- 
lengths. 

Observations of high redshift galaxies carried out in 
the NIR offer significant advantages over optical observa- 
tions: extinction due to local dust is smaller in the NIR; k- 
corrections are smaller, and therefore less critical; adaptive 
optics is easier to achieve in the NIR. However, there are 
also significant disadvantages in NIR astronomy, the chief 
concern being the high background level. Table 1 lists typi- 
cal surface-brightnesses for a good observing site, as used in 
the ESO ELT exposure time calculator 3 . 

At wavelengths longer than the H band, thermal emis- 
sion from the telescope and surroundings dominate the back- 
ground emission. Therefore the detector and camera must 
be designed to operate at cryogenic temperatures - a signif- 
icant technical challenge. At shorter wavelengths the ther- 
mal background is less problematic but instead there is the 
supplementary frustration of very bright hydroxyl emission 
(Meinel 1950; Dufay 1951), resulting from the vibrational 
decay of an excited OH-radical, formed from the combina- 
tion of a hydrogen atom with an ozone molecule. Further- 
more the intensity of the OH emission is highly variable (see 
section 2.1.2 below), making accurate sky subtraction very 
difficult (e.g. Davies 2007). 

The intensity and variability of the NIR background 
have traditionally restricted the depth of NIR observations 
compared to optical observations. However, two attributes 

2 http://www.so.stfc.ac.uk/roadmap/rmQucstionsHome.aspx 

3 http: / / www.eso.org/observing/etc/bin/ut3 / conica/script/eltsimu 



of the NIR background spectrum suggest that in principle 
it should be possible to obtain very deep observations in the 
NIR. Firstly the OH spectrum comprises nearly all (« 99.9 
per cent from our model, see § 2) of the J and H band 
background. The interline continuum, due mainly to zodi- 
acal scattered light, is expected to be very faint, e.g. the 
models of Kelsall et al. (1998) give a flux density of only 
w 77 photons s _1 m~ 2 arcsec~ 2 fim -1 on the ecliptic plane. 
Secondly, the OH lines are intrinsically very narrow (Turn- 
bull & Lowe 1983), meaning that large sections of the NIR 
sky spectrum will be very dark if the OH lines can be effi- 
ciently removed, thus enabling deep observations. 



There are several methods which can, in principle, ex- 
ploit the dark interline continuum for deep NIR observa- 
tions, including narrow-band filters which allow observa- 
tions of a very short bandpass between lines (e.g. DAzLE; 
Horton et al. 2004), or space-based telescopes (e.g. JWST; 
Gardner et al. 2006). Here we discuss ground-based OH 
suppression, by which we mean the selective removal (by 
means of low transmission, reflection etc.) of very narrow re- 
gions of the spectrum corresponding to the OH lines, whilst 
maintaining high throughput in the interline regions. There 
have been several methods proposed to achieve OH sup- 
pression, including spectroscopic masking at high disper- 
sion (e.g. Maihara & Iwamuro 2000), rugate filters (Offer & 
Bland-Hawthorn 1998) and multi-notch holographic filters 
(Blais-Ouellette et al. 2004). We discuss these methods in 
section 5.1; each offers potential improvements over natural 
conditions, but in practice there are problems in achieving 
efficient OH suppression by any of these means. By efficient 
we mean achieving high suppression of the lines whilst main- 
taining high interline throughput, for a large number of lines 
(;> 100). In this paper we make the case for using fibre Bragg 
gratings (FBGs, see Bland-Hawthorn et al. 2004) to achieve 
OH suppression (§ 5.2); FBGs do not suffer from any of the 
drawbacks that have prevented successful application of the 
methods mentioned above. There is now a real case for be- 
lieving that highly efficient OH suppression is close to being 
demonstrated. 



Before discussing methods of OH suppression, we first 
review the background sources contributing to the NIR in 
section 2. We then address the scattering properties of dis- 
persive optics (§ 3), the proper understanding of which is 
essential to achieve efficient OH suppression. The study of 
scattering shows that the OH lines must be suppressed prior 
to dispersion; the combination of the extremely bright OH 
lines and the strong scattering wings, that are an unavoid- 
able consequence of any diffraction system, make attempts 
to remove or 'resolve out' the OH skylines after dispersion 
futile. We then make the case for OH suppression, i.e. se- 
lectively removing the OH lines prior to dispersion (§ 4), 
including a review of previously proposed methods (§ 5.1), 
and their limitations. We then show that fibre Bragg grat- 
ings offer the prospect of cleanly and efficiently removing OH 
lines photonically (§ 5.2). We calculate the required level of 
suppression and spectral resolution to achieve scientifically 
useful observations. We summarise our case in section 5.4, 
and conclude with some remarks on the future in section 6. 
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Figure 2. A lower limit on the wavelength at which the OH 
emission dominates the NIR background as a function of envi- 
ronmental temperature. This was calculated as the wavelength at 
which the blackbody spectrum from a typical telescope exceeds 
that of the zodiacal scattered light at fi = 60 degrees. 



2 THE NIR BACKGROUND 

We now consider the various elements that contribute to the 
NIR background, focussing on the wavelength range 0.9- 
2.0/im, as the region in which OH suppression will be most 
effective. Following Tokunaga in chapter 7 of Cox (2000) , wc 
divide the NIR background into the following components: 
the OH airglow; the thermal emission from the ground based 
telescope; the atmospheric thermal emission; the zodiacal 
scattered light; zodiacal emission; galactic dust emission; the 
cosmic microwave background radiation. Each of these will 
be discussed in turn. Our final model of the NIR background 
spectrum is shown in Figure 9. 



2.1 Hydroxyl emission lines 

2.1.1 Emission mechanism 

The OH emission (Meinel 1950; Dufay 1951) is by far the 
largest component of the NIR background at A <J 2.0/im. At 
longer wavelengths the background is dominated by ther- 
mal emission, from the telescope and dome etc. The exact 
wavelength where the thermal emission becomes dominant 
depends on the temperature of the environment, as illus- 
trated in Figure 2, which shows the wavelength at which the 
blackbody spectrum from the telescope exceeds the black- 
body spectrum of the zodiacal scattered light, i.e. a lower 
limit to the wavelength at which OH emission dominates 
the background. Thus for very cold sites, such as Dome C 
in Antarctica with an average year round temperature of 
w 220-K" (see Burton et al. 2005) OH suppression would be 
effective at wavelengths A <C 2A^m. 

The OH emission arises mainly from the combination of 
hydrogen and ozone from a layer about 9km thick at an alti- 
tude of around 87km. This reaction leaves a rotationally and 



vibrationally excited OH molecule (and an oxygen molecule) 
with a radiative lifetime of a few milliseconds (Content 1996, 
and references therein). The emission then results from the 
vibrational decay of the molecule, i.e., 



H + 3 -> OH* + 2 . 



(1) 



The intrinsic width of the lines is determined from the 
Doppler shift due to the thermal motions of the molecules, 
from pressure broadening, and from natural broadening. The 
former effect dominates the inner width of the line and has a 
Gaussian form, whereas the latter has a Lorentzian profile, 
giving the lines broad wings, albeit very faint ones. 

The Doppler width can be estimated from the following 
argument. For a gas in thermal equilibrium the gas particles 
will have a distribution of speeds, but the most probable 
speed will be when the kinetic energy, mv 2 /2 is equal to 
the thermal energy kT. Thus the light emitted from the 
gas particle will be Doppler shifted by an amount A A/ A = 
±\vr\/c. Thus, 



AAdb = — 



2A 2kT 



(2) 



where m is the mass of the particles, and T is the gas temper- 
ature. Thus for OH molecules at a temperature of T — 200K 
(Rousselot et al. 2000), AAdb ~ 5 x 10~Vm. 

Pressure broadening is the result of interactions or col- 
lisions with neighbouring particles in the gas. The effect can 
be estimated by considering the average time between col- 
lisions or encounters, Ato, which is approximately equal to 
the mean free path divided by the average speed of the par- 
ticles, i.e. 



At 



ry / 2kT/m' 



(3) 



where a is the collisional cross section and n is the number 
density. It is assumed that the shape of the line is Lorentzian, 
which is the result of an emitting atom undergoing simple 
harmonic motion. Substituting equation 3 into equation 7 
yields, 



AA P 



A ncr c 



C7T 



2kT 
m 



(4) 



We adopt Aa pb ~ 3 x 10 V m i an upper limit based on 
the difference in wavenumber of < 0.002 cm -1 quoted by 
Turnbull & Lowe (1983). 

Natural broadening is due to the Heisenberg uncertainty 
in the particles energy, 



At' 



(5) 



where At is the duration of the change in energy, such that, 



AAa 



(6) 



2-KcAt ' 

The full calculation shows that the FWHM is in fact twice 
this value, 

A 2 



AAf 



TTcAt ' 



(7) 



Thus for a vibrationally excited state lasting a few millisec- 
onds, the natural broadening is w 5 x 10~ 13 /im at A = 1/jm. 
From the above estimates the Doppler broadening is by 
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Figure 3. The theoretical logged Voigt line profile due to Dopplcr 
and pressure broadening, of a very bright OH line line with an 
integrated flux of 740 7 s^ 1 m -2 arcsec -2 assuming T = 200K 
and 7 = 1.5 X 10~ 13 m. The Lorentzian wings are significant out 
to at least lA from the line centre. 



far the most dominant contribution to the intrinsic width 
of the line. However, due to the much broader wings of 
the Lorentzian pressure broadening, this mechanism is not 
negligible, as it will contribute much more to the region 
between the line peaks. The natural broadening can be 
safely neglected. Therefore in modelling the OH emission 
line spectrum we include both the Doppler broadening and 
the Lorentzian profile, i.e. we use a Voigt profile, 



V(a,j,x) 



aV2^ it{(x - x') 2 + 7 2 ) 



da;', 



(8) 



where a — 2v / 2 " B 2 and AAdb is the full-width at half max- 
imum of the Doppler broadened line given by equation 2, 
and 7 is the half-width at half maximum of the pressure 



broadened line given by 



in equation 4. 



In our models we assume a temperature at 87km of 
200K and 7 = 1.5 x l(n 13 m. The resulting Voigt profile is 
shown over a range of ±lCT 3 ^im in Figure 3 for the brightest 
doublets in the H band (which are not resolved) . The plot is 
logged to emphasise the Lorentzian wings which are signifi- 
cant out to w 5 x W~ 4 (im either side of the line centre (c.f. 
zodiacal scattered light described below). However in prac- 
tice the interline continuum is dominated by instrumental 
scatter. We include this effect in our model, but in practice 
line widths are dominated by the instrumental profile (see 
§ 3) and doublet separations. 

Note that although the intrinsic width of any individual 
line is extremely narrow, as shown in Figure 3, the OH spec- 
trum is actually composed of close doublets. The doublets 
are of equal intensity (Osterbrock et al. 1996) and arise due 
to A-type doubling (see Herzberg 1939) from the interaction 
between the rotation of the O and H nuclei and the orbital 
angular momentum of the electron. The practical line width 



Figure 4. The distribution of doublet separations for OH lines 
with wavelength 0.9-1.8/im. 



is determined by the separation of these doublets. Using 
the models of Rousselot et al. (2000), we have determined 
the doublet separation for all lines in the wavelength range 
0.9 ^ 1.8, shown in Figure 4. The distribution is strongly 
peaked at separations of 0.1-0.2 A, which gives a practical 
limit to the line widths. Note though, that we model the OH 
spectrum by treating each line individually. 



2.1.2 Variability 

The OH emission is not only extremely bright, but it is also 
highly variable in the spatial and temporal domain. This 
has significant consequences for the removal of the night sky 
background from observations. Despite its strong variability 
a few trends with time and space are known. The intensity 
of the lines increases with zenith distance due to the longer 
path length through the emitting layer of the atmosphere. 
The intensity changes as, 



ije) 

1(0) 



, _ I rsing \ 2 
_ \r + h) 



(9) 



where 9 is the zenith angle, 1(9) is the intensity at 9, r is 
the radius of the Earth and h is the altitude of the emitting 
layer above the ground (Content 1996). 

The temporal variation is due to two main causes: diur- 
nal variation in temperature, and density waves in the upper 
atmosphere. 

The diurnal variation has been modelled by Shimazaki 
& Laird (1970), taking into account reactions between 14 
molecular species and diffusion. We show their model in 
Figure 5, and summarise their description of the variations. 
The features result from two emission layers, a broad layer 
at 45km altitude and a narrow layer at 80km. The height 
of the lower layer rises in the afternoon, and partially dis- 
appears around sunset, causing a dip in the OH emission. 
About twenty minutes after sunset, both layers rise and the 
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Figure 5. The night time variation of the model of Shimazaki & 
Laird (1970) by the continuous line, and our analytic approxima- 
tion by the dotted line. 

intensity of the emission is greatly enhanced due to an in- 
crease in the ozone content at sunset; this causes the sudden 
rise in the models. After sunset the lower layer disappears 
completely because of the removal of hydrogen, and the up- 
per layer decreases slowly due to the decrease of O3. There is 
also a rapid dimming and brightening around sunrise. This 
model is consistent with observations, which show a decrease 
by a factor ~ 2 in the OH emission throughout the night 
(Ramsay et al. 1992; Content 1996). The model is well ap- 
proximated by the function 1/(1 + t) ' 57 , where t is the time 
in hours, as shown in Figure 5. We assume that a typical 
nights observing lasts for 8hrs, so for observations longer 
than than this we reset the OH brightness every 8hrs. 

Aside from this long term decline in OH line strength 
throughout the night, there is also short term variability 
caused by density waves (Ramsay et al. 1992; Frey et al. 
2000). These are typical periodic variations of length 5-15 
minutes, with an amplitude of about a tenth of the flux. 

We assume that all OH lines vary in unison for our 
simulations. In fact the real situation is more complicated; 
lines within the same transition will be correlated, but sep- 
arate transitions will not. However, to a reasonable approx- 
imation, lines from a particular transition all lie within the 
same region of the spectrum, with very little contribution 
from other transitions (Davies 2007). Hence the assumption 
of unison will be true for sections of spectrum ~ 0.1/im long. 

2.1.3 Line list 

Rousselot et al. (2000) have modelled the emission spec- 
trum of the night-sky OH lines, giving accurate wavelengths 
for 4732 lines, and approximate relative intensities. In or- 
der to use this atlas we have normalised the theoretical rel- 
ative intensities to the fluxes measured by Maihara et al. 
(1993b) (after first removing the O2 lines). The lines were 
measured with a resolving power of R « 17, 000 and re- 




Theoretical relative line strength 

Figure 6. The correlation between the theoretical relative line 
strengths of Rousselot et al. (2000) and the measured line 
strengths of Maihara et al. (1993b). The closed symbols are for 
the J band and have a slope of 1.4 X 10~ 5 . The open symbols 
are for the H band, with a slope of 3.7 X 10~ 6 . The dashed line 
between the J and the H band correlations is the geometric mean 
of 7.2 X 10~ 6 , which was used to calibrate the Rousselot list. 

mained unresolved. Since the OH lines are close doublets 
we combine all theoretical lines which have a separation of 
AA < A/17000, before comparing the theoretical and ob- 
served line strengths. Figure 6 shows the correlation be- 
tween the theoretical line strengths and the measured line 
strengths. There is a significant difference between the J and 
H band data, and the H band data shows a stronger corre- 
lation. This is indicative of the transient nature of the OH 
lines. We have used the geometric mean of the J and H band 
correlations to calibrate the Rousselot data. 

2.2 Other lines 

There are a number of O2 and N2 lines in the J band that 
are not included in the models of Rousselot et al. (2000). 
These lines are self-absorbing and dim rapidly after sunset 
(Content 1996). We have included line strengths for these 
taken directly from Maihara et al. (1993b). For simplicity, 
we have assumed they are broadened according to exactly 
the same prescription as the OH lines. 

2.3 Thermal emission from the telescope 

There will be significant thermal emission from the telescope 
and the instrument employed. The instrument design can be 
optimised to cool as many parts as possible thereby minimis- 
ing its contribution to the background, but some parts, e.g. 
optical fibres may be external to the main instrument dewar 
and therefore unable to be cooled. Even so, the main con- 
tribution will come from the much larger telescope, and we 
model the emission as a black-body spectrum at T = 273 K, 
with an emissivity of e = 0.02 (chapter 7, Cox 2000), i.e., 
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2hc 2 



A 5 (ef^r - l) 
which can be expressed as a surface brightness, 

he 

1.41 x 10 16 e 



(10) 



(11) 



XUie^^ - 1) 
in units of photons s _1 m~ 2 nmT 1 arcsec -2 . 

2.4 Zodiacal scattered light 

Zodical light is emission from sunlight scattered by inter- 
planetary dust. As such it has a solar spectrum, multiplied 
by the scattering efficiency as a function of wavelength. Indi- 
vidual measurements in the optical (e.g. Levasseur-Regourd 
& Dumont 1980) and the NIR (e.g. Noda et al. 1992; Mat- 
sumoto ct al. 1996) seem to show a solar spectrum when 
considered individually, but have different normalisations 
when compared, with the NIR zodiacal spectrum being 
twice as bright due to the increased scattering efficiency off 
larger dust particles at longer wavelengths (Matsumoto et al. 
1996). 

The brightness of the zodiacal light obviously decreases 
as a function of ecliptic latitude. Kelsall et al. (1998) have 
modelled the zodiacal light in detail in order to remove this 
component when analysing data from the COBE Diffuse In- 
frared Background Experiment (DIRBE). We approximate 
the Kelsall model with a Lorentzian function, which enables 
a faster calculation whilst maintaining most of the accuracy. 
In Figure 7 we show their model's dependence on ecliptic 
latitude, f3, for 1.25pim emission and our fit to it given by, 



0.75 



(l + (ol3) 2 ) 



+ 0.25, 



(12) 



where (3 is in radians. 

For the majority of astronomical applications of FBGs 
we anticipate a preference for high galactic latitudes, and 
therefore a wide range of ecliptic latitudes depending on the 
galactic longitude, which could change the zodiacal bright- 
ness by a factor of ~ 2. 

For the purposes of the model we normalise a solar 
spectrum (using equation 11 with e = 1.08 x 10~ 13 and 
T = 5800K) to the Kelsall et al. (1998) model value for the 
ecliptic plane. Several measurements and models of the zo- 
diacal scattered light exist in the literature. These are com- 
piled in Table 2. Note that while the models predict the zo- 
diacal scattered light should be fainter at longer wavelengths 
(since it has the form of a T = 5800K black-body) the mea- 
sured data disagree with this. Note too, that the ground 
based measurements are rather higher than the models. In 
section 3 we suggest that for the ground based measurements 
scattering of OH light may have a significant contribution 
to these values. 



2.5 Moonlight 

Moonlight is known to be an insignificant source of back- 
ground with conventional NIR observations. This is because 
the OH skylines dominate over scattered moonlight and also 



E 0.6 




/?/ radians 

Figure 7. The variation of zodiacal light with ecliptic latitude 
from Kelsall et al. 1998 shown by the continuous line and our 
model as given by equation 12 shown by the dotted line. 



because Rayleigh and Mie scattering are both less efficient 
at longer wavelengths; Rayleigh scattering has a A~ 4 depen- 
dence and Mie scattering has a A -13 dependence. However 
in the regime of OH-suppressed observations these assump- 
tions need to be re-evaluated. 

Krisciunas & Schaefer (1991) present a model of the 
brightness of scattered moonlight that is accurate to 23 per 
cent. However, this model is only appropriate for observa- 
tions in the V band. Therefore we have adopted the proce- 
dure described in the CFHT Redeye Manual 4 to adapt the 
model to NIR wavelengths. The principal changes are al- 
lowing for the change in colour assuming a solar spectrum, 
and allowing for the wavelength dependence of the scattering 
function. We show the results of the model in Figure 8. Com- 
paring to estimates of the zodiacal scattered light shows that 
for observation at a full, or nearly full, moon, then scattered 
lunar light would be significant out to w 50 degrees from 
the moon. OH suppressed observations should therefore not 
be performed around the full moon, though 'grey nights' 
should be acceptable. We note that the effect of scattered 
moonlight could have had an effect on previous attempts to 
measure the interline continuum, if these were performed in 
bright time and the contribution due to moonlight was not 
taken into account. 



2.6 Other components 

There are a number of other components that make im- 
portant contributions to the IR background at longer wave- 
lengths (> 2/im), but are negligible in the regime considered 
here. These are: the thermal contribution from the atmo- 
sphere, zodiacal emission from interplanetary dust at the 



Sec http: / / www.cfht . hawaii . cdu /Instruments /Detectors /IR /Redeye /Manual /cha] 
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Table 2. Measurements and model predictions of the zodiacal scattered light compiled from the literature. "Includes 
instrumental background. 6 Model based on COBE DIRBE measurements. c From a STSI report by M. Stiavelli, see 
http: / / www.stsci.edu/hst / nicmos/documcnts /handbooks / currcnt_NEW/c04_imaging8.html#309385 
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Figure 8. The brightness of scattered moonlight as a function of 
angular distance away from the moon. The different curves are 
for different lunar phases from to 180 in steps of 20 degrees. It 
is assumed that the moon is at the zenith. 



ecliptic pole, Galactic emission from from interstellar dust, 
the cosmic background radiation. 

It is possible that there is also a faint continuum or 
pseudo-continuum composed of line emission from NO2 or 
other atmospheric models (Content 1996). Such emission 
could perhaps explain some of the discrepancy between mea- 
surements of the zodiacal scattered light and the ground 
based measurements of the interline continuum. However 
there are no reliable measurements of this continuum, since 
previous estimates do not take into account the instrumen- 
tal scattering profile (§ 3), therefore we do not include any 
such continuum in our model. 



2.7 Atmospheric transmission 

The dominant absorption features of NIR light are those 
due to water, carbon dioxide, nitrous oxide, methane and 
ozone in the atmosphere, which define the J, H and K band 
photometric windows. We have used the model transmission 
profile at Mauna Kea assuming an airmass of 1.0 and a Wa- 
ter vapour column of 1.6mm in our default sky background 
model (Lord 1992, obtained from Gemini Observatories 5 ). 
See Figure 9. 



3 SCATTERING OF DISPERSIVE OPTICS 

Because the OH lines are so bright compared to the interline 
continuum (see Figure 9) they cannot be cleanly removed 
through high dispersion spectroscopy. This is because even 
a small amount of scattered OH light will swamp the true 
interline continuum. To see this, consider the example of a 
ruled diffraction grating. The theoretical normalised inten- 
sity distribution of a grating with N lines is given by, 



I = h 



where, 



sin/3 



(sinA r 7 \ 
iVsury J 



(13) 

(14) 
(15) 



H = ysinc9, 

kh . 
7 = — sm<9, 

and k is the wavenumber, b is the slit width, h is the slit 
separation, and 9 is the angle through which the light is 
diffracted. 

Woods et al. (1994) show that equation 13 may be rep- 
resented as a Lorentzian plus a constant background, 

Y «^ (x-^ + ^ +B > < 16 > 

where Yet is the ratio of intensity at wavelength A to inten- 
sity at Ao, B is a constant background term due to Rayleigh 
scattering and 

Ao 



JVefv/2' 



(17) 



http://www.gemini.edu/sciops/ObsProcess/obsConstraints/ocTransSpcctra.htm] 
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Figure 9. The NIR background model. The top panel shows the atmospheric transmission. The middle panel show the contributions 
from the OH spectrum (blue), the zodiacal scattered light (red), the telescope thermal emission (black). The bottom panel shows the 
final resulting spectrum. 



The Case for OH Suppression 9 



and it is assumed that A — Ao << Ao and N > 100. Equa- 
tion 16 is normalised by a factor 1/(tvlo + 2hB), where h is 
the groove spacing. 

The most important factor in determining the line con- 
tribution is N c the number of effective lines. In theory the 
number of effective lines should be the number of lines illu- 
minated. However, Woods et al. (1994) find that N e is al- 
ways measured to be much lower than this theoretical limit. 
In practice iV c can be taken to be the smallest number of 
consecutive grating grooves with no imperfections. Woods 
et al. (1994) report values that are typically 0.25-0.5 the 
theoretical upper limit, but occasionally as low as 0.16. Fur- 
ther from the line centre the constant background Ab also 
becomes important. 

Woods et al. (1994) fit profiles to 10 different types of 
gratings including holographically and mechanically ruled 
gratings and find the scattered light is indeed well fit by 
a Lorentzian plus constant. Our own measurements with 
IRIS2 (a grism spectrometer) and AAOmega (a VPH spec- 
trometer) arc spectrum confirm this; we find that the spec- 
tral line profile can be fit by a function, 



(*-A r 



I = Io 



(l-o)/<rV21og(2) 



2^ a 7r{(A-A„)2 + 2/2 ( 72 1og(2)} 



,(18) 



which is simply a Gaussian plus a Lorentzian, with a con- 
tribution of factor a from the Gaussian and 1 — a from the 
Lorentzian, and the Lorentzian has FWHM / times larger 
than that of the Gaussian. We find typical values of / = 4-6 
and a = 0.89-0.97. In Figure 10 we show a fit to the IRS2 
and AAOmega arc spectra. 

We now show the effects of these scattering wings on the 
OH spectrum, and in doing so we address the large discrep- 
ancy between the ground-based observations of the interline 
continuum listed in Table 2 and the model values. We show 
the effect of scattering on the OH lines around A = 1.665/im, 
where Maihara et al. (1993b) measured the interline contin- 
uum. 

We have modelled the scattering from the University 
of Hawaii 2.2m Coude spectrograph, with the 600 lines/mm 
grating, and Abiazc = 1.32^im used for the H band observa- 
tions by Maihara et al. (1993b). We have assumed B = 10 s , 
the value for the Bausch & Lomb grating used by Woods 
et al. (1994). 

We use equation 16 to model the scattering profile as 
it can be normalised exactly. Note that this will provide a 
slightly conservative estimate of the scattering far from the 
line centres, as the wings in equation 16 drop off faster than 
in the full equation. 

In Figure 11 we show the scattered light from all H 
band lines reported in Maihara et al. (1993b) in the win- 
dow 1.660-1.675 (c.f. their figure 2) whence the continuum 
emission was measured. It is unclear what the theoretical 
value of iV c should be, i.e. the number of illuminated lines, 
but it will be much less than 120, 000 (e.g. Woods et al. 
1994). The real value of N e , i.e. the smallest number of con- 
secutive lines without imperfections will be smaller again. 
Therefore we show plots for four different values of N e : for 
N = 120,000 = 20cm x600 lines mm" 1 , i.e. the absolute 
maximum number of lines, and for values of N e = 60, 000, 
30,000 and 15,000. The scattered wings are higher for fewer 
lines. These estimates give a scattered light intensity of 




(a) AAOmega, a = 0.97 and / = 5.5. 




(b) IRIS2, a = 0.89 and / = 4.0. 

Figure 10. The line profiles of two spectrographs, AAOmega, 
a VPH spectrograph and IRIS2 a grism spectrograph. The mea- 
sured data arc shown by the open circles, and a fit of the form of 
equation 18 is shown by the line. 



1.665/im. Thus it is possible that scattered light could ac- 
count for a significant part of the difference between our 
predicted value of the continuum at A = 1.665/im and that 
measured by Maihara et al. (1993b), if the number of im- 
perfections in the grating is high. 

The scattering behaviour of dispersive optics thus pre- 
vents observations reaching the level of the interline contin- 
uum regardless of instrument background or spectral resolu- 
tion. For example in Figure 12 we show the relative strengths 
of the OH spectrum with a resolution of R = 10000 (using 
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Figure 11. The scattered light due to all measured OH lines in 
the H band. The different curves are for values of N = 120, 000, 
60,000, 30,000 and 15,000. The scattered wings arc higher for 
fewer lines. 
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Figure 12. The relative strength of the OH spectrum at R = 
10000 (thin line) compared to the zodiacal scattered light at (3 = 
60 degrees (thick line). The scattered OH line light reaches the 
interline continuum for 11 per cent of the spectrum. 

a scattering profile of the form equation 18 with a = 0.90 
and / = 4) and the interline continuum at an ecliptic lati- 
tude p = 60 degrees over the range 0.9 ^ A ^ 1.8/im. The 
interline continuum is reached for only 11 per cent of the 
spectrum between 0.9 < A < 1.8/im, and 3 per cent between 
1.4 ^ A ^ 1.8/nm. 



4 THE CASE FOR OH SUPPRESSION 

By OH suppression we mean the selective removal of OH 
lines from the incident light whilst allowing light either side 
of the deleted line to pass through with high efficiency (as 
opposed to OH avoidance by narrow band filters e.g. Hor- 
ton et al. 2004, which selects light between OH lines, and 
is therefore restricted to narrow bandpasses). We discuss 
how this may be achieved using fibre Bragg gratings below 
(§ 5.2). First however, we address the desired performance 
of a general OH suppression system. 

The OH spectrum contains hundreds of bright lines 
(Rousselot et al. 2000; see § 2). An OH suppression system 



must remove a section of finite wavelength, AA, from the 
incident spectrum for each line to be suppressed. After sup- 
pression, these regions will be unsuitable for any scientific 
analysis. Therefore OH suppression comes with the penalty 
of reducing the useful bandpass of the spectrum, which will 
vary with the number of lines suppressed and the size of 
the 'notches', AA. In practice the regions around bright OH 
lines are not usually suitable for scientific analysis in any 
case, since the variation of the OH emission hampers the 
success of sky-subtraction and residual sky lines are usually 
observed. 

In designing an OH suppression system it is thus neces- 
sary to choose how many lines to remove and at what reso- 
lution in order to maximise the efficiency of the system. We 
now address this issue for notches fixed at AA =lA, 3A and 
10A (A/AA w 10000, 3000 and 1000), for varying numbers 
of lines. The lines are suppressed in order of their relative 
brightness, with the brightest lines removed first. We mea- 
sure the efficiency of the system by the fraction of useable 
spectrum remaining and the brightness of the faintest line 
suppressed (as an estimate of the expected sensitivity of the 
observations) . Note that in practice many OH lines are very 
close doublets, and therefore these can be removed with a 
single lA notch. 

Figure 13 shows the results for up to 200 notches across 
the J and H bands. Even with 200 lines of width lA, sup- 
pressing down to a level of 4 x 10 -3 photons s _1 mT 2 
arcsec~ 2 fim^ 1 (far fainter than would practically be nec- 
essary), only 8 per cent of the J band has been removed, 
and only 7 per cent for 200 lines in the H band. Thus, 
by virtue of the intrinsic slenderness of the OH-emission 
lines, suppression at high resolution is a highly desirable ob- 
jective, maintaining a large fraction of scientifically useful 
wavelength range, whilst vastly increasing signal to noise. 



5 METHODS OF OH SUPPRESSION 

There are several methods which have been proposed, and in 
some cases employed, to achieve OH suppression. However, 
to date no instruments exist which can suppress OH light 
at high efficiency whilst maintaining good interline through- 
put. In the next section we will review proposed methods of 
OH suppression and address their limitations. This will be 
followed by a discussion of FBGs, in which we will argue that 
there is now a real case for believing that highly efficient OH 
suppression will finally be acheived. 

5.1 Previous methods of OH suppression 

5.1.1 High dispersion masking 

Maihara et al. (1993a) present a method of OH suppression 
using high dispersion and a ruled mirror. The basic idea 
is to disperse the light at high resolution, whereupon it is 
reflected from a mirror-mask, which has a series of black 
lines coincident with the position of the OH lines, which 
are therefore not reflected. The reflected light, with the OH 
lines removed, then proceeds back through the camera and 
grating system in the opposite direction to its initial journey, 
and hence is recombined into a 'white-light' beam. This final 
beam may then be dispersed at low resolution as required. 
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Figure 13. The effect of increasing the number of OH suppressing notches on the incident spectrum. The left panels show results from 
the J band and the right the H band. The top panels show the fraction of the total passband that remains usable after suppression. The 
bottom panels show the flux of the faintest line suppressed. 



This method of mechanically removing the OH light and 
recombining is preferable to using software to achieve the 
same result, e.g. by removing the OH lines, then rebinning 
the spectrum, because the read-noise becomes much more 
significant for a highly dispersed spectrum (Iwamuro et al. 
1994). 

Spectrographs employing this technique have been built 
and used on the Subaru telescope, e.g. OHS+CISCO (Iwa- 
muro et al. 2001; Motohara et al. 2002), and on the Univer- 
sity of Hawaii 2.2m (Iwamuro et al. 1994). These instruments 
succeeded in reducing the cores of the OH lines to « 4 per 
cent of their original strength. However, it is the reduction 
in the background between the lines that is important in as- 
sessing the merit of an OH suppression system, since this is 
where scientific measurements will be made. Thus the fun- 
damental limit to the performance of OHS-style suppressors 
is the scattering of the OH light. This process was described 
in detail in section 3. Figure 11 illustrates the problem: the 
Lorentzian scattering wings prevent the interline continuum 
ever being reached. This is true even at high resolution as 
shown in Figure 12. Furthermore, in a mirror-mask system 
the problem of scattered light is compounded, since scatter- 
ing will occur for each pass through the grating system, and 
from diffraction from the mirror-mask itself. The scattered 
wings of the OH line remain in the spectrum since only the 
core of the line is suppressed with the mirror mask; between 
the lines the gains will be much less. To realise efficient OH 



suppression this scattering problem must be avoided; the 
OH light must be removed prior to dispersion. 

The same principle of a mirror-mask design is being 
used in FMOS (fibre multi-object spectrograph; Maihara 
et al. 2000), which should begin use on the Subaru telscope 
in 2008. The performance of the OH suppressing part of 
the FMOS instrument is expected to be similar to OHS, 
but with the advantage of multiplexed observations and im- 
proved spectrographs. 

5.1.2 Rugate filters 

Rugate filters have been proposed by Offer & Bland- 
Hawthorn (1998) as a method of OH suppression applicable 
to imaging. Rugate filters consist of a transparent substrate 
onto which is deposited a thin film of dielectric material. By 
varying the composition of the film as it is deposited the re- 
fractive index can be made to vary throughout the coating. 
By controlling the refractive index variation any arbitrary 
transmittance profile may be designed, e.g. by combining 
various sinusoids using Fourier theory. 

The chief obstacle in realising OH suppression with ru- 
gate filters has been the difficulty of manufacture. Deposi- 
tion of each layer of the film requires very tight tolerances 
in the thickness of the layer, which must be uniform over 
the entire area of the substrate. These difficulties have so 
far made manufacture of satisfactory filters impracticable. 
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A further problem is that even should the tolerances of each 
layer be met, thousands of layers are required which leads 
to a coating which is tens of microns thick. The stresses in 
each layer of so many coatings ultimately makes each layer 
unstable (Ennos 1966). 

5.1.3 Holographic filters 

Blais-Ouellette et al. (2004) suggest the use of volume phase 
holographic filters (Barden et al. 2000) as a method of OH 
suppression. These filters use holographic recording tech- 
niques to record a grating consisting of fringes of varying 
refractive index in a volume of transparent material, typi- 
cally dichromated gelatin. If the fringes are arranged to be 
parallel to the surface of the filter, no dispersion occurs, and 
the grating acts as a notch filter. By controlling the fringe 
pattern, several notch filters may be multiplexed within a 
single filter. Blais-Ouellette et al. (2004) demonstrate a fil- 
ter capable of suppressing 10 lines (or close doublets) at 
lOdB, with a notch width of lA and an interline throughput 
of 85 per cent. 

These filters have the advantage as a viable method 
of OH suppression in that they are currently available for 
purchase from Photon Etc. However, there are some diffi- 
culties in their use in a practical situation. The wavelength 
response of the filter profile shifts as a function of off-axis 
angle, which restricts the use of the filters to slow beams. In 
order to use the filters in slow beams, the notch width must 
be made larger to accommodate the shifting filter profile, 
reducing their efficiency. Another issue is that a single filter 
is made up from several glass substrates which are cemented 
together such that the layers between each substrate aligns 
with the optical axis. To avoid scattering between the layers 
the boundaries must be blackened, which therefore divides 
the beam before it enters the VPH. 

A comparison of VPH filters with FBGs strongly 
favours the latter. The relatively small number of lines per 
filter compared with FBGs, coupled with the lower interline 
throughput is disadvantageous. FBGs are able to suppress 
150 lines across a 400nm window with a interline through- 
put of 88 per cent. The same 150 lines would require 15 of 
the filters demonstrated by Blais-Ouellette et al. (2004), or 
5 of the 30 line filters advertised by Photon Etc. This would 
result in a throughput of 9-44 per cent. 

FBGs can suppress the OH lines at 30dB, whereas the 
VPH filters suppress filters at lOdB. This makes a significant 
difference in the performance of the two devices. In the next 
section we show how the suppression factor strongly influ- 
ences the performance of FBGs (Figure 14) . If the OH lines 
are not suppressed strongly enough, the residual lines will 
still dominate the background signal due to the scattering 
effects described above. 

5.2 Fibre Bragg gratings 

Bland-Hawthorn et al. (2004) present a radical new ap- 
proach to OH suppression employing fibre Bragg gratings. 
Fibre Bragg gratings (FBGs) are optical fibres with a slowly 
varying refractive index in the core. As light propagates 
down the fibre it suffers Fresnel reflections due to the vary- 
ing refractive index. By properly controlling the index vari- 
ation it is possible to produce strong reflections at specific 



wavelengths. Bland-Hawthorn et al. (2004) showed it was 
possible to produce a low loss filter defined by a series of ir- 
regularly spaced, narrow notches with suppression depths as 
high as 30 dB, whist maintaining high throughput outside of 
the notches (losses^ 0.03 per lOOnm wavelength coverage). 
Their design corrects for 18 OH lines in the H band. 

Two steps are required in order for this technology to be 
useful for astronomical observations: the ability to suppress 
many lines, and the ability to work with multi-mode fibres. 
These two requirements are not easily compatible. When one 
considers the huge complexity of suppressing up to 150 OH 
lines within the H band, the amplitude and phase structure 
of the grating demands that this is printed into a single- 
mode fibre so that the ~ 10° cycles are available to pro- 
duce the filter response along one propagation axis (Bland- 
Hawthorn et al. 2008). In a multimode fibre, a much larger 
number of cycles is required to achieve the same grating 
response to each mode, thereby greatly increasing the com- 
plexity of the FBG. Thus FBGs achieve peak performance 
within a single-mode fibre (10 micron core), whereas astro- 
nomical fibres, with their larger core diameters, are highly 
multi-moded. Therefore a successful implementation of pho- 
tonic OH suppression requires that the many modes of the 
input fibre be transformed into the same number of single 
modes acting in parallel. 

To this end, we developed the photonic lantern (Leon- 
Saval et al. 2005, Bland-Hawthorn 2005) which decomposes 
the spatial modes of the input fibre into a parallel degenerate 
array of single-mode fibres. Identical gratings are printed 
into each of the fibres. Once the filtered light exits the fibre 
array, these are recombined by an identical taper back to a 
multimode output fibre. Horton & Bland-Hawthorn (2007) 
have demonstrated that fibres with as few as 5 modes can 
have high coupling efficiency, paving the way for AO fed 
FBGs in the near future. 

The combination of these two innovative technologies 
means that the development of FBGs for astronomical pur- 
poses is now at a stage that is compatible with astronomi- 
cal requirements. FBGs are now being developed to remove 
~ 150 lines in each of the J and H bands. The brightest lines 
are suppressed at 30dB, and fainter lines are suppressed by 
an amount proportionally less. Photonic lanterns are be- 
ing developed with 37 fibre tapers. The resulting multimode 
FBG will be a complex device, and expensive to manufac- 
ture, at least in the current incarnation. Initially, only tens 
to hundreds of fibres are envisaged, but ultimately, the tech- 
nology will undergo further development to bring down the 
unit cost. 

In light of the discussion in section 3, an important 
advantage of FBGs is that they can be incorporated into 
a spectrograph before the dispersive elements, thus the OH 
light is strongly rejected with very little scattering, offering 
for the first time the capability to achieve extremely deep 
NIR observations without the requirement to put a telescope 
and instrument into space. 

The ultimate goal of OH suppression is to reach the 
interline continuum over a significant fraction of the J and 
H bands. The level of suppression required to achieve this 
depends on the spectral resolving power of the instrument. 
Figure 14 shows the fraction of the 1.4-1.8/nm window in 
which the OH spectrum falls below the zodiacal scattered 
light as a function of suppression and spectral resolution. A 
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Figure 14. The fraction of the 1.4-1.8/xm window over which 
observations can reach the interline continuum as a function of 
maximum suppression. Spectral resolutions of R = 250, 500, 1000, 
2000, 3000, 4000 and 5000 are shown by the different colours. 
The hatched curves show the range for the Kelsall et al. (1998) 
and NICMOS models at an ecliptic latitude of [3 = 60 degrees. 
The dotted lines are for the interline continuum of Maihara et al. 
(1993b). 



maximum suppression of 40dB yields significant gains, and 
thereafter the improvements flatten off. This is true of both 
the high interline continuum of Maihara et al. (1993b) and 
the models. 



5.3 Simulations 

In order to assess the expected performance of FBGs we 
have thoroughly modelled an entire observing system from 
astronomical source to the detector, incorporating an instru- 
ment employing FBGs. Three different source types were 
chosen to illustrate the power of OH suppression with FBGs, 
a z = 11 QSO, a z = 3 galaxy and a faint T dwarf. 



5.3.1 System model 

The simulations work by starting with a fluxed Id object 
spectrum and modifying it according to background contri- 
butions and losses for each component of the system. A sim- 
plified schematic overview of a typical simulation is shown 
in Figure 15, showing the sources of emission and losses we 
have modelled. Many of the components shown are optional, 
for example, we can also model galaxy, and stellar spectra, 
amongst other sources. It should also be noted that most 
of the components shown are fully adjustable, e.g. OH line 
strengths, zodiacal scattered light etc. can all be adjusted ac- 
cording to meaningful parameters. The observation param- 
eters and mode of observation are then applied to generate 
the final spectrum. 

The details of the simulations were as follows. The zodi- 
acal background is described above (§ 2.4), and we modelled 
observations at /3 — 60 degrees. The OH line spectrum was 
used as described in section 2.1, including temporal varia- 
tion. The atmospheric absorption is described in section 2.7. 
We assumed an 8m telescope with obstruction losses of 5 per 
cent was employed in Cassegrain focus with a reflectivity of 



97 per cent off both mirror surfaces. We modelled the ther- 
mal emission from the telescope as a T — 273K blackbody 
with an emissivity of 3 per cent. 

The FBGs are modelled as 150 line gratings of lA 
width. The fibres are taken to have a core size of 25/im, 
i.e. few mode fibres, corresponding to an angular diameter 
of 0.2 arcseconds at f/5. The fibre coupling losses are ~ 0.15 
for a few mode fibre of this size (Horton & Bland-Hawthorn 
2007). These are fed with a generic AO system, achieving a 
Strehl ratio of 0.3, with a throughput of 0.9. The fibre aper- 
ture losses are thus, « 46 per cent, due to lost light, assuming 
that the PSF can be modelled as a diffraction limited PSF 
plus a Gaussian seeing disc with FWHM 0.5 arcsec, where 
the two components are normalised such that the diffraction 
limited part contains a power equal to the Strehl ratio. 

The spectrograph is assumed to have a throughput of 
35 per cent, and an adjustable resolution. The sampling is 
assumed to be 3 spatial by 3 spectral pixels. A systematic 
wavelength calibration error of a tenth of a pixel FWHM 
has been included. We assume a blocking filter is used at 
the entrance to the spectrograph and apply the correspond- 
ing transmission. The detector is taken to have an effective 
read noise of 4e~ when up-the-ramp sampling is employed, 
and a dark current of two thousandths of an electron per 
pixel per second, typical of Rockwell Hawaii-2RG HgCdTe 
arrays (Smith et al. 2006). Observations are composed of 
30 minute reads, with sky-subtraction performed by cross- 
beam switching between pairs of fibres. 



5.3.2 General results 

The system described above results in a reduction of the to- 
tal background by a factor of of 42 and 17 in the H and J 
bands respectively, corresponding to 4 and 3 magnitudes. It 
is important to realise that this reduction applies across the 
entire spectrum including the interline regions. For example 
at A = 1.665^im (where Maihara et al. 1993b measured the 
interline continuum) the background would be dominated 
by scattered OH light in the absence of FBGs. However, 
because FBGs suppress the OH lines before scattering, the 
interline background is also reduced by a factor of 40. We 
emphasise that it is the magnitude of the interline reduc- 
tion that is scientifically important, and that the advan- 
tage of FBGs over other methods of OH suppression is their 
ability to suppress the OH light before the light enters the 
spectrograph, and therefore before any significant scattering 
surfaces are encountered. Methods to remove OH light after 
dispersion may achieve a good overall reduction in the over- 
all background due to the removal of the cores of the bright 
emission lines (as could also be achieved through software 
suppression), but between the lines, where the science will 
be done, the reduction will not be as good due to the scat- 
tered light. 

We compare the reduction in background resulting from 
OH suppression to that resulting from AO correction in Fig- 
ure 16. Changing from a 0.5 arcsec aperture for natural see- 
ing to a 0.2 arcsec aperture for AO correction results in a 
reduction of the background by a factor 6.25. Applying OH 
suppression then results in a further reduction of a factor of 
42 in H or 17 in J as described above. Given that diffrac- 
tion limited observations would severely over-sample a high 
redshift galaxy of size 1 arcsecond, we argue that the main 
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Figure 15. A much simplified schematic overview of a typical simulated spectroscopic observation, showing the main causes of emission 
and losses, with which we have modelled the performance of the fibres. 
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Figure 16. Simulated spectra of a z = 3 H = 19 mag galaxy, 
observed at a resolution of R = 1000 for 1 hr. These spectra 
are of a very bright galaxy, and have no sky-subtraction in or- 
der to emphasise the equivalent width of the OH lines compared 
to the continuum. The thick black line shows the actual object 
spectrum; the red line shows the object plus background for the 
galaxy observed with AO and FBGs as described in section 5.3.1; 
the blue line is for AO correction only; the green line is for natural 
seeing of 0.5 arcsec. 

advantage from AO for cosmology is the reduction in back- 
ground light. Consequently we argue that OH suppression 
is at least as important for cosmology as AO, since it pro- 
duces a greater level of background suppression. Of course 
both technologies must proceed together, but given the sig- 
nificant advances promised by OH suppression we argue that 
the development of FBGs should be given a very high prior- 
ity for future instruments on existing telescopes and on the 
ELTs of the future. 



5.3.3 QSO simulation 

The typical surface brightness of sources responsible for 
reionisation is AB=27-29 mag arcsec 2 , achievable by a 
source of 26 AB mag per square arcminute if the number 
density is « 0.1-10 arcrnin" 2 (Stiavelli et al. 2004). This 
source density is reasonable considering the number counts 
of Yan & Windhorst (2004) for z ~ 6 galaxies, which have 
~ 0.3 galaxies per square arcmin at AB=26 mag in ap- 
proximately the same waveband. We have therefore assumed 
an unabsorbed magnitude of AB=26 mag, or H = 24.6 
Vega mag (using the conversion given by Tokunaga & Vacca 
2005). 

We used a simulated z — 10, H — 20 mag spectrum 
(provided by Richard MacMahon, private communication), 
which includes Gunn & Peterson (1965) absorption and 
metal line absorption. We then redshifted this to z = 11 
(assuming no further absorption due to neutral hydrogen), 
and normalised the H band flux to 24.6 mag. Figure 17 
shows a simulation of a 70hr exposure at R = 1000. The 



bottom panel shows the results of an identical system with- 
out FBGs, and is dominated by residual sky lines. The use 
of FBGs results in a remarkable improvement in the qual- 
ity of the resulting spectrum. The Gunn & Peterson (1965) 
absorption trough is clearly visible when FBGs are used, 
whereas without FBGs the spectrum is completely domi- 
nated by noise and residual sky lines. FBGs offer the chance 
to accurately measure the redshifts of extremely faint ob- 
jects during the epoch of reoinisation. This will be discussed 
further in the summary. 

5.3.4 Redshift 3 galaxy simulation 

We have simulated an observation of an H = 23 Vega mag 
galaxy at z — 3. The exposure time was 8hrs and the reso- 
lution was R = 1000. The galaxy spectrum is from the syn- 
thetic spectral libraries of Bruzual & Chariot (2003), and 
assumes an instantaneous burst of star-formation, an age 
of lGyr and solar metallicity. Figure 18 shows the result- 
ing spectrum as observed through a system with FBGs and 
a system without FBGs. The FBGs result in a high qual- 
ity spectrum with Ca H and Ca K absorption lines clearly 
visible. Thus FBGs offer the chance to obtain precise mea- 
surements of galaxies when the Universe was only a fifth of 
its present age, allowing a detailed study of evolution over 
almost the entire history of galaxies. 

5.3.5 T dwarf simulation 

Cool stars emit most of their radiation in the NIR and are 
identified by molecular absorption features. The faintest T 
dwarf to date has a magnitude of J = 20.28 Vega mag (Za- 
patero Osorio et al. 2002; see Adam Burgasser's T dwarf 
page 6 ), however most known T dwarfs have magnitudes 
around J ~ 15-16 Vega mag, and are located within a few 
tens of parsecs from the Sun. 

Figure 19 shows a simulated spectrum of a J = 21 Vega 
mag T dwarf observed at R — 1000, in J and H simula- 
taneously, for 8hrs. The spectrum is that of a T5 dwarf, 
2MASS0559-1404 (McLean et al. 2003 7 ) scaled to J = 21. 
The FBG spectrum is of much superior quality to the spec- 
trum employing ordinary fibres. FBGs offer the opportunity 
to identify much fainter cool stars than ever before. 

5.4 Summary 

We have demonstrated the possibility to achieve extremely 
deep near-infrared observations using fibre Bragg gratings. 
Hitherto, attempts to obtain deep NIR observations have 
been frustrated by the extremely bright and variable OH 
emission from the atmosphere. We have shown that in or- 
der to efficiently remove OH lines from astronomical spectra 
the OH lines must be suppressed prior to the light enter- 
ing the camera. Scattering from dispersive optics results in 
Lorentzian line profiles, the wings of which are much brighter 
than the true interline continuum, and in most cases are 
brighter than the sources being observed. The variability of 
the OH lines makes it extremely difficult to subtract cleanly. 

6 http:/ /web. mit.edu/ajb/www/tdwarf/ 

7 http:/ /www. astro. ucla.edu/~mclean/BDSSarchive/ 
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Figure 17. A simulated spectrum of an H = 24.6 Vega mag, z = 11 QSO as observed by the system described in § 5.3.1. The exposure 
time was 70hr, and the spectral resolution was R = 1000. The top panel shows a system with FBGs and the bottom panel shows an 
identical system without FBGs. The red lines indicate the true object spectrum. 



FBGs can suppress OH lines at high resolution, R 
1 0. 000, high suppression (30dB) and with high throughput 
out of the suppression regions (« 90 per cent). Suppressing 
150 lines in the J or H band, with a maximum suppression of 
30dB, decreases the sky background by a factor of « 45, or 4 
magnitudes. Achieving this will have the potential to extend 
NIR observation to objects orders of magnitude fainter than 
currently possible. The resulting observations will have a 
great impact on many areas of astronomy, and we highlight 
a few now. 



5-4-1 First light and reionisation 

FBGs offer the opportunity to study the rest-frame UV light 
from objects at the time of reionisation. Figure 17 shows a 
simulated spectrum of a z — 11 QSO with a clearly visible 
Gunn & Peterson (1965) absorption trough. Spectroscopy 
with FBGs would therefore be ideally suited to identifying 
first light sources, performing follow-up spectroscopy from 
surveys of Lyman-a emission (e.g. Taniguchi et al. 2005), 



continuum drop-out due to the Lyman-break (e.g. Stanway 
et al. 2004) and ultra-narrow band surveys (e.g. Horton et al. 
2004). A spectrum of a complete Gunn- Peterson trough 
would confirm the redshift of the source, which would allow 
estimates of the luminosity function and number-magnitude 
relations of the first sources of light in the Universe. If the 
sources are bright enough, then it may be possible to also 
measure the line profile of Lyman-a which is expected to be 
distinctively asymmetrical due to resonant scattering blue- 
wards of the line-centre. The intrinsic width of the line it- 
self will depend upon the type of source; quasars will have 
broad Lyman-a lines, FWHM« 1500km s _1 , whereas galax- 
ies will have narrower lines, FWHM« 100km s _1 . The dif- 
ference in line widths leads to the possibility of distinguish- 
ing the type of sources responsible for ionisination. For the 
purpose of confirming the presence of a redshifted Lyman-a 
line it is necessary to resolve the lines sufficiently to measure 
the asymmetry, requiring resolutions of R > 300000/100 = 
3000. 

The combination of FBGs and ELTs will make it possi- 



The Case for OH Suppression 17 




1.4 1.5 1.6 1.7 

X/ fj.m 



Figure 18. A simulated spectrum of an H = 23 Vega mag, 2 = 3 galaxy as observed by the system described in § 5.3.1. The exposure 
time was 8hr, and the spectral resolution was R = 1000. The top panel shows a system with FBGs and the bottom panel shows an 
identical system without FBGs. The red lines indicate the true object spectrum. 



ble to thoroughly explore the ionisation history of the IGM 
using NIR spectroscopy of first light objects. The detection 
of complete Gunn-Peterson absorption troughs in the spec- 
tra of high redshift quasars (Becker et al. 2001) show that 
the IGM was partly neutral at z ~ 6.5. However, a neutral 
fraction of only xm ~ 10~ 4 is all it takes for complete ab- 
sorption due to the high oscillator strength of the Lyman-a 
transition. Thus to probe further back in time to uncover 
the details of the ionisation of the IGM, we must look to 
other means. 

Oh (2002) suggest that metal absorption lines may pro- 
vide a useful means of measuring the ionisation state of the 
IGM. Ol and Sin have ionisation potentials close to that of 
neutral hydrogen, and thus measuring the neutral fraction 
of either of these metals could be used to infer the neu- 
tral fraction of hydrogen. Furthermore there are useful lines 
just longer than Lyman-a so a single measured spectrum 
may incorporate Lyman-a 1215. 67A, Ol 1302. 17A and Sin 
1260. 42A. These metals have previously been observed in 
the spectra of high redshift quasars (Pettini 2001) verify- 



ing their usability for this purpose. Since it is unlikely that 
metals will be smoothly distributed through the IGM, the 
signature will be in the form of a forest of lines rather than 
a Gunn-Peterson trough. A measurement of the equivalent 
width of the lines, or even simply the number of lines will 
provide precious information on the ionisation state. 

The detection or non-detection of a radiation damp- 
ing wing has the potential to divulge important information 
on the ionisation state of the IGM. If a radiation damp- 
ing wing is found it will provide a very accurate measure of 
the neutral fraction of hydrogen (Miralda-Escude 1998). If 
the ionising sources reside in significant cosmic Stromgren 
spheres then a damping wing will be impossible to measure 
(Cen & Haiman 2000; Madau & Rees 2000). However, the 
line profile may still be identified as Lyman-a due to its 
asymmetrical shape, and thus the lack of a damping wing 
can place constraints on the ionising fluxes of the sources. If 
the sources are galaxies then it may also place constraints 
on the clustering of the objects, since it is unlikely that a 
single primordial galaxy would produce a significant ionised 
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Figure 19. A simulated spectrum of a J = 21 Vega mag,T5 dwarf as observed by the system described in § 5.3.1. The exposure time 
was 8hr, and the spectral resolution was R = 1000. The top panel shows a system with FBGs and the bottom panel shows an identical 
system without FBGs. The red lines indicate the true object spectrum. 



region on its own. If spectra can be obtained with a suf- 
ficiently high signal-to-noise, then the tail of the damping 
wing may provide detailed information on the size of the 
Stromgren spheres. 

Figure 20 shows the same simulation as described in 
section 5.3.3, except the telescope was assumed to have a 
diameter of 30m. Metal absorption lines are now visible in 
the spectrum. 



5.4.2 Galaxy evolution 

Deep NIR imaging and spectroscopy with FBGs will allow 
galaxy evolution to be studied over most of the history of 
the Universe. Figure 18 shows a H — 23 Vega mag galaxy 
at z — 3. Absorption lines are clearly visible. It would thus 
be possible to measure galaxy masses, luminosities and star- 
formation rates from emission lines, and to derive the evolu- 
tion of the star-formation history, the luminosity function, 
the Faber & Jackson (1976) relation and if high spatial res- 
olution imaging is available, the fundamental plane. 



5.4-3 Cool stars 

NIR infrared observations are essential for identifying cool 
stars such as L, T and Y dwarfs. Measuring the number den- 
sity of such objects as a function of mass is a critical test 
of our understanding of both star and planet formation pro- 
cesses. NIR spectroscopy is essential for fully characterising 
their temperature, surface gravity and atmospheric condi- 
tions. The use of FBGs would allow observations of objects 
orders of magnitude fainter than currently possible (see Fig- 
ure 19), thus extending the survey volumes and numbers of 
such objects, and placing strong constraints on the low mass 
end of the initial mass function. 



6 REALISATION 

The 8- 10m telescope generation has been with us for more 
than a decade, and astronomy must now plan for a new 
generation of facilities if the field is to advance in the years 
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Figure 20. A simulated spectrum of an H = 24.6 Vega mag, z = 11 QSO as observed by the system described in § 5.3.1 with a 30m 
telesscope. The exposure time was 70hr, and the spectral resolution was R = 1000. The top panel shows a system with FBGs and the 
bottom panel shows an identical system without FBGs. The red lines indicate the true object spectrum. 



to come. For ground-based observations, the Earths atmo- 
sphere provides a major challenge to scientific progress on 
two fronts: (a) the movement of air cells blurs the cosmic 
signal; (b) hydroxyl emission from the upper atmosphere 
produces a perpetual twilight, thereby greatly increasing the 
background signal in astronomical observations. At near in- 
frared wavelengths, there are clear indications that adaptive 
optics is overcoming the problem of seeing and is starting 
to deliver unprecedented resolution and sensitivity at near- 
infrared wavelengths (Liu 2006). In contrast, the extreme 
brightness of the infrared sky remains an unsolved problem. 

The James Webb Space Telescope (Gardner et al. 2006) 
will be launched sometime in the next decade, bypassing al- 
together the problems associated with observing through the 
atmosphere. However, it will have a 6m aperture compared 
to apertures of 20-42m of the next generation of ground- 
based telescopes. The satellite will be inaccessible and will 
not be serviced by astronauts, thus the instrument suite will 
have restricted functionality compared to what is possible 
today on ground-based sites. For these reasons it is impor- 



tant to continue to address the issue of the infrared night 
sky background for ground-based telescopes. 

The AAO has spearheaded the development of a new 
technology that promises to completely solve the infrared 
twilight problem, i.e. by removing 99 per cent of the OH 
line emission with an overall efficiency of 80 per cent or bet- 
ter. The concept exploits two recent advances in photonics: 
(i) broadband fibre Bragg gratings (FBGs); (ii) multimode 
to single-mode photonic lanterns. While the development 
remains challenging, there are indications that the develop- 
ment is on track to solve the OH suppression problem once 
and for all. We have demonstrated the extraordinary sci- 
entific potential of these fibres in this paper. Note that all 
simulations include correction by adaptive optics, for both 
cases, with and without FBGs. The observations we have 
presented and discussed would not be possible through use of 
adaptive optics alone. The improvements over the AO-only 
observations demonstrate a compelling need for FBGs, if 
such deep observations are to be achieved. The development 
and implementation of FBG fed instruments on existing 8m, 
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and especially on future ELTs, deserves equal attention to 
that which has been invested in the development of adaptive 
optics. 

In subsequent papers, we will discuss the relative mer- 
its of OH suppression in natural seeing compared to cor- 
rected seeing through adaptive optics, and describe instru- 
ment concepts that are under development to capitalize on 
the photonic OH suppression development. 
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